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FOREWORD 


Personal  computer  and  video  games  have  shown  dramatic  improvements  in 
processing  speed  and  graphics  capability.  As  a  result,  the  game  representation  of  military 
activities  has  increased  in  realism,  leading  to  questions  about  their  potential  use  in  Army 
training.  These  questions  focus  on  identifying  and  quantifying  the  effects  of  games  and  their  use 
in  skill  acquisition,  retention,  and  transfer  of  training  to  Army  tasks.  The  research  reported  in  this 
note  addresses  whether  or  not  exemplar  military  games  provide  doctrinally  correct  approaches  to 
conducting  military  individual  and  collective  tasks.  The  work  builds  on  previous  STRICOM 
efforts  to  identify  a  specific  approach  for  evaluating  PC  games  in  terms  of  ease  of  application 
and  appropriateness  for  incorporation  into  Army  training.  The  findings  from  this  research  are 
intended  to  be  used  to  evaluate  and  recommend  PC  games  for  incorporation  in  Army  institutional 
and  field  training  programs,  based  on  their  characteristics  and  instructional  value. 

The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  Simulator  Systems 
Research  Unit,  conducts  and  sponsors  research  with  the  goal  of  providing  information  that  will 
improve  the  effectiveness  of  training  simulators  and  simulations.  The  work  described  here  is  a 
part  of  ARI  Research  Task  202a,  VERITAS  -  Virtual  Environment  Research  for  Infantry 
Training  and  Simulation.  This  work  was  performed  by  the  Institute  for  Simulation  and  Training, 
University  of  Central  Florida. 


STEPHEN 


Acting  Technical  Director 
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LOW-COST  PC  GAMING  AND  SIMULATION  RESEARCH:  DOCTRINAL  SURVEY 


EXECUTIVE  SUMMARY 


Research  Requirement: 

The  U.S.  Army  Simulation,  Training  and  Instrumentation  Command  (STRICOM)  has 
been  investigating  personal  computer  and  video  games,  generically  referred  to  as  PC  games,  as 
low  cost/complexity  training  devices.  One  aspect  of  the  program  was  to  determine  whether  low- 
cost  PC  games  could  be  used  as  viable  training/educational  interventions  that  achieve  measurable 
learning  outcomes.  That  training  analysis  identified  general  cognitive  skills  that  were  addressed 
by  a  wide  range  of  games.  A  review  of  the  cognitive  skills  analysis  identified  the  doctrinal 
correctness  of  game  parameters  and  playing  patterns  as  critical  to  effective  transfer  of  skills  from 
games  to  military  activities.  The  current  effort  assessed  the  doctrinal  correctness  of  two  PC 
games  (used  in  the  previous  investigation),  as  a  step  toward  providing  methods  for  evaluating 
and  recommending  PC  games  for  incorporation  in  Army  institutional  and  field  training  programs 
based  on  game  characteristics  and  instructional  value. 


Procedure: 

Two  PC  games,  previously  analyzed  for  potential  application  in  military  skills  training, 
were  investigated  for  doctrinal  correctness  within  potentially  applicable  occupational  specialties. 
A  portion  of  the  previous  analytical  process  was  adapted  for  conducting  subject  matter  expert 
evaluations  addressing  the  doctrinal  correctness  of  armor  and  infantry  off-the-shelf  games  for 
training.  The  games  were  strategically  mapped  into  detailed  documents  that  portrayed  game 
elements,  options,  and  missions  in  a  clear  fashion  that  allowed  canned  game  scenarios  to  be 
mapped  to  military  training  procedure,  protocol,  and  fidelity.  These  materials  were  then 
presented  to  available  subject  matter  experts  in  a  focus  group  type  of  evaluation,  moderated  by 
the  researcher,  and  directed  toward  attaining  information  on  the  game's  potential  for  training  or 
practice  in  the  context  of  doctrinal  correctness. 


Findings: 

Overall,  Rogue  Spear:  Covert  Ops<>"  (an  infantry  or  special  operations  game)  was  judged 
as  providing  users  with  a  realistic  and  doctrinally  correct  simulation  of  many  facets  of  covert 
operation  missions.  The  infantry  SMEs  especially  liked  the  opportunities  to  practice  doctrinally 
correct  aspects  of  room  and  building  clearing;  for  example  formations  at  the  entrance  to  a  room, 
and  the  ability  to  move  systematically  throughout  a  building.  Rogue  Spear:  Covert  Ops«">  also 
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offered  an  excellent  multiplayer  cooperative  capability  that  was  judged  as  enabling  the  practice 
of  team  skills  among  individuals.  An  evaluation  of  Steel  Beaststm  (a  tank  battle  game)  resulted 
in  the  SMEs  deciding  that  despite  the  many  strong  elements  in  the  game,  it  had  facets  that  could 
potentially  result  in  negative  training  or  cause  doctrinal  confusion  on  the  part  of  soldier  students, 
Discussions  about  pending  update  releases  of  the  game  (which  would  include  Ml  A2 
simulations)  were  encouraging,  as  that  equipment  was  the  basis  for  many  negative  comments. 


Utilization  of  Findings: 

The  U.  S.  Army  will  employ  PC  gaming  technology  for  training,  rehearsal,  and 
evaluation  both  in  local  and  distributed  formats.  A  usable,  easy  to  implement  process  could 
prove  to  be  extremely  beneficial  to  military  agencies  that  were  considering  integrating 
commercial  off-the-shelf  (COTS)  PC-based  synthetic  environments  into  training  programs.  The 
game  mapping  process  described  in  this  effort,  as  well  as  the  evaluation  process,  are  techniques 
that  would  be  effective  in  identifying  what  game  elements  are  candidates  for  training 
applications.  Clearly  elements  of  the  examined  games  can  be  useful,  especially  when  organic 
mission  editors  can  be  used  to  tailor  the  relevant  portion  of  the  games  to  satisfy  real  training 
requirements.  Feedback  from  the  SMEs  indicates  that  with  a  mission  editor  and  a  “guide  for 
instructors”  the  games  would  be  useful  additions  to  training  programs.  Games  can  provide  a 
more  engaging  instructional  approach  than  is  currently  being  used  at  the  installations  with  which 
the  SMEs  are  familiar.  The  potential  usefulness  of  COTS  PC  games  should  be  studied  further 
with  soldiers  engaged  in  training  programs,  and  assessed  against  training  requirements. 
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Introduction 


The  U.S.  Army  Simulation  Training  and  Instrumentation  Command  (STRICOM) 
established  a  research  program  investigating  novel  techniques  for  low  cost/complexity  training 
devices,  design  strategies  for  improving  the  acquisition  and  retention  of  individual  and  team 
skills,  and  measuring  the  effectiveness  of  instructional  features.  Under  this  effort,  the  University 
of  Central  Florida  (UCF),  Institute  for  Simulation  and  Training  (1ST)  began  researching  low-cost 
Personal  Computer  (PC)  gaming  and  simulation  to  determine  whether  low-cost  video  games 
could  be  used  as  actual  viable  training/educational  interventions  that  achieve  measurable 
learning  outcomes.  The  results  of  that  research  provided  a  direct  link  between  the  simulation  and 
training  industry  and  the  video  game  industry.  The  conclusions  provided  guidelines  for  the 
inclusion  of  new  cost-effective  game  media  within  military  training  that  could  substantially 
increase  learning  efficiency  and  performance. 

The  initial  research  investigated  and  documented  the  learning  potential  of  PC-based 
games  for  the  larger  simulation  and  training  community.  The  research  focused  on  military 
training  for  individuals  and  teams,  primarily  in  the  area  of  cognitive  skills,  such  as  procedures, 
planning,  decision-making,  etc.  The  1ST  research  effort  analyzed  data  from  PC-based  gaming 
subjective  responses  and  developed  basic  cognitive  templates  that  use  game  features  and  options 
to  select  various  PC-based  games.  Additionally,  the  data  provided  initial  support  for  an 
intemalization/extemalization  gaming  theory  (Morris  &  Tarr,  2002). 

The  effort  documented  in  this  report  used  subject  matter  expert  (SME)  evaluations  to 
assess  the  doctrinal  accuracy  of  PC  games  for  training.  The  evaluations  were  conducted  by 
defining  game  parameters  and  environments  and  then  obtaining  the  subjective  opinions  of  SMEs 
regarding  existing  doctrinal  policies  or  tactical  techniques  within  the  activities  of  the  game.  1ST 
used  subjective  SME  reports  to  establish  the  procedure,  instruments,  and  techniques  essential  to 
measure  the  degree  of  doctrinal  correctness.  To  clarify  the  concept  of  doctrinal  evaluation,  the 
following  examples  are  offered: 

•  The  typical  approach  for  dismounted  soldiers  moving  through  urban  terrain  is  to  use 
bounding  overwatch  while  moving  close  to  walls  and  maintaining  adequate  cover.  If  a  game 
did  not  support  teams,  one  concern  would  be  that  proper  bounding  overwatch  could  not  be 
reinforced  in  the  trainees.  If  the  game  rewarded  players  for  rapid  movement  rather  than 
cautious  and  covered  approaches,  the  game  might  create  inappropriate  tactical  patterns  in  the 
users  that  could  be  detrimental  if  transferred  to  field  exercises. 

•  Deviations  in  weapons  systems  capabilities  and  military  symbology  might  also  lead  to 
incorrect  transfer  and  inappropriate  tactical  doctrine  during  operations. 

•  Interaction  between  the  players  and  the  Semi-automated  Forces  (SAF)  or  Artificial 
Intelligence  (AI)  might  lead  to  lowered  or  narrowly  patterned  opposing  force  expectations  by 
the  trainee. 
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There  are,  however,  multiple  dimensions  to  PC  games.  For  example,  they  might  train  or 
reinforce  the  right  kinds  of  cognitive  skills  for  a  small  unit  leader  but  do  so  in  a  context  that 
reinforces  incorrect  doctrine.  One  consequence  might  be  a  game-induced  error  in  tactics  on  the 
battlefield,  although  the  leader  gained  excellent  decision-making  skills  by  playing  the  game. 

PC  Gaming  and  Training  Literature  Review 

Research  conducted  in  the  early  part  of  the  decade  indicated  that  computer  games  and 
simulations  were  low  on  fidelity,  interface,  and  immersion  dimensions  (Thurman  &  Mattoon, 
1994).  Compared  to  the  PC  games  of  today,  they  could  not  faithfully  represent  the  real  world 
nor  did  they  have  very  natural  interfaces,  and  the  feeling  of  being  immersed  in  the  simulation 
was  accomplished  only  through  imagination.  All  that  has  changed  with  the  significant  strides  we 
have  seen  in  the  development  of  graphics  and  animation  software,  AI,  computer  interfaces,  and 
networking.  Just  as  graphics  and  computer  interface  advances  have  produced  more  realistic 
simulations  of  the  physical  world,  AI  is  resulting  in  more  realistic  modeling  of  game  characters. 
AI  at  the  human  level  can  be  applied  to  tactical  enemies,  partners,  support  characters,  and  even 
units  of  individuals,  such  as  a  platoon  (Laird  &  van  Lent,  1999). 

The  availability  of  networked  simulation  technology  has  allowed  the  training  of  general 
team  skills  such  as  aircrew  resource  management  behaviors  (Baker,  Prince,  Shrestha,  Oser,  & 
Salas,  1993),  air  combat  (Huddlestone,  Harris,  &  Tinsworth,  1999)  and  commercial  barge  and 
tugboat  crew  skills  (Salopek,  1998).  This  provides  mission  rehearsal  capability  which  can  be 
embedded  where  appropriate,  to  provide  training  anywhere,  anytime.  Commercial  off-the-shelf 
(COTS)  personal  computer  and  console  games  (referred  to  generically  as  PC  games)  now  also 
conform  to  many  technology  and  development-based  standards  such  as  being  Distributed 
Interactive  Simulation  (DIS)  Compliant,  so  they  may  be  linked  together  in  advanced  training 
scenarios  across  the  spectrum  of  operations. 

PC  Games 

The  PC  gaming  and  simulation  industry,  largely  driven  by  recent  technology  advances 
and  consumer  economics,  has  dramatically  driven  cost  down  while  improving  the  quality  and 
realism  of  games  and  desktop  simulation  technologies.  The  cost-capability  curve  available 
today  was  unheard  of  just  two  years  ago.  Several  agencies,  including  different  branches  of  the 
US  military,  are  exploring  the  possible  use  of  PC  games  as  a  supplement  to  some  aspects  of 
training.  The  initial  investigations  are  focused  on  training  that  uses  expensive  real  world 
exercises,  or  costly  simulation  technology  applications  such  as  head  mounted  display  (HMD) 
virtual  environments.  Specific  features  of  PC  games  are  also  being  investigated  as  practice  and 
feedback  alternatives  or  classroom  enhancements.  The  goal  is  determining  low-cost  training 
alternatives  for  assignments  that  don’t  easily  allow  required  job  or  skill  training,  such  as 
assignments  to  Bosnia  or  on  board  ship. 

Recent  advances  in  PC  technology,  such  as  high-speed  processors,  expanded  memory, 
and  high-performance  video  cards  with  3D  capability  have  made  high  quality  synthetic 
environments  technology  inexpensive.  Additionally,  COTS  game  developers’  use  of  reputable 
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military  data  sources  for  game  models  have  made  these  games  increasingly  attractive  to  the 
military  for  inclusion  in  training  (Coleman  and  Johnston,  1999).  For  example,  there  has  been 
strong  acceptance  of  the  Center  for  Naval  Education  and  Training  (CNET)  plans  to  implement  a 
formal  training  program  around  the  Microsoft  Flight  Simulation  98/2000“"  software  (Dunlap  & 
Tarr,  1999;  Koonce  &  Bramble,  1998).  The  military  is  also  evaluating  other  games  as  potential 
low-cost  flight  simulators  (Coleman  &  Johnston,  1999). 

Applying  PC  Games 

Games  and  simulations  have  always  been  a  part  of  education  and  learning  strategies, 
especially  in  military  training.  PC  games  provide  the  opportunity  for  knowledge  or  skills  to  be 
acquired  and/or  practiced  in  a  variety  of  settings  and  contexts  so  that  they  may  be  understood, 
integrated,  and  accessible  in  future  situations  (Ruben,  1999).  This  type  of  environment  is 
important  today  because  a  very  large  number  of  military  deployments  and  Operations  Other  than 
War  (OOTW)  cause  military  personnel  to  move  out  of  their  basic  warfare  operations  and  into 
situations  where  there  is  little  or  no  way  to  keep  tactical  skills,  especially  cognitive  skills, 
current. 

Educational  games  have  also  been  favorably  compared  to  classroom  instruction  for  the 
teaching  of  social  studies,  math,  language  arts,  logic,  physics  and  other  sciences  (Randel,  Morris, 
Wetzel,  &  Whitehill,1992).  Simulation  building  games  are  suggested  to  be  an  effective  training 
tool  for  teaching  urban  geography  and  urban  planning  because  they  add  motivation  to  the 
learning  process  (Adams,  1999).  In  other  game  types,  players  are  engaged  in  competitive 
interactions  in  which  they  follow  a  set  of  rules  to  achieve  specified  goals  that  depend  on  skill  and 
often  involve  chance  and  are  potentially  highly  engaging  and  motivating.  Business  simulations 
and  scientific  simulations  were  found  to  be  acceptable  teaching  tools  in  classroom  settings 
(Ruben,  1999).  Commercial  uses  of  simulation  are  prominent  in  the  medical  community,  NASA, 
nuclear  power,  and  commercial  aviation  (Thurman  &  Dunlap,  1999).  Additionally,  the  military 
uses  the  majority  of  simulator-based  training  programs.  PC  games  have  also  been  shown  to 
enhance  soldiers’  decision-making  skills  by  providing  practice  with  variation  (Riddle,  1997). 
Similarly,  (Pillay,  Brownlee,  &  Wilss,  1999;  Ricci,  Salas,  &  Cannon-Bowers,  1996) 
demonstrated  that  players  practiced  complex  cognitive  processes  such  as  interpreting  explicit  and 
implicit  information,  inductive  reasoning,  metacognitive  analysis,  and  problem  solving.  Homan 
(1998)  also  suggested  that  low-cost  low-fidelity  PC  aviation  simulations  can  be  used  to  enhance 
pilots’  situational  awareness. 

PC  games  have  an  array  of  applications  and  provide  easy  and  engaging  opportunities  for 
practice  and  skill  retention  of  those  more  abstract  but  critical  thinking  and  decision-making 
skills.  Games  and  simulations  may  not  appear  to  be  similar  to  instructional  techniques,  but  as 
learning  environments  they  have  overlapping  characteristics.  Both  are  examples  of  experiential 
instructional  methods  in  that  they  are  interactive  and  foster  active  learning  (Ruben,  1999). 
According  to  Brown  (1999),  both  require  a  temporary  suspension  of  disbelief  as  participants 
accept  a  false  situation  as  temporarily  real.  Their  differences  lie  in  how  players  participate.  In 
training  simulations,  players  participate  in  situations  or  processes  in  order  to  learn  about  specific 
real-world  settings  or  procedures.  Recent  studies  have  suggested  that  PC  simulation  games  can 
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produce  a  general  transfer  of  cognitive  skills  that  have  application  to  a  wide  variety  of  domain- 
specific  tasks.  Other  studies  have  used  recent  PC  games  for  conducting  psychological  research 
on  the  cognitive  processes  involved  in  problem  solving,  strategy  development  (Gonzales  & 
Cathcart,  1995)  and  changes  in  neural  maps  (Tallal,  Miller,  Bedi,  Byma,  Wang,  Nagarajan, 
Schreiner,  Jenkins,  &  Merzenich  1998). 

In  summary,  PC  games  now  have  the  capability  to  assist  learning,  transfer,  and 
performance  in  a  variety  of  domains,  including  substitution  for  real  world  training  requirements. 
Since  PC-based  technology  is  at  a  point  where  human  inclusion  or  immersion  is  fundamental,  the 
capability  and  feasibility  of  applying  PC  games  to  enhance  performance,  training,  and 
educational  utility  is  evident.  The  question  becomes  how  to  select  and  use  specific  games  or 
portions  of  games  for  specific  training  requirements. 

Project  Objective 

In  order  to  apply  PC  COTS  games  to  military  training  one  needs  an  analytical  approach 
to  analyze  and  extract  components  of  PC  games  for  customized  application  capability.  This 
effort  addressed  one  part  of  this  need  by  investigating  the  development  of  one  component  of  an 
iteratively  developing  systematic  model  for  analysis  and  certification  of  low-level  COTS 
synthetic  environments  for  training  (Morris  &  Tarr,  2002).  Obviously,  time  and  resources  were 
too  limited  for  full-scale  training  program  development  investigations.  Therefore,  a  general 
analytic  approach  was  used  to  identify  and  investigate  the  games  potential  alternative  learning 
value  to  live  or  expensive  large-scale  simulation  applications.  The  eventual  goal  is  a  modeled 
process  that  researchers  and  developers  could  use  to  reduce  costs  of  new  instruction,  software 
development  and  non-recurring  systems  engineering. 

The  immediate  objective  for  this  effort  was  to  focus  on  the  evaluation  of  doctrinal 
correctness  or  veracity  of  potential  PC  games  by  conducting  interviews  with  Army  standards 
SME.  During  this  process,  training  application  and  usefulness  were  also  assessed  as  resources 
and  time  allowed.  A  portion  of  the  previous  validation  model  process  was  adapted  for 
conducting  SME  evaluations  regarding  the  doctrinal  accuracy  of  armor  and  infantry  off-the-shelf 
games  for  training  (Morris  &  Tarr,  2002).  The  analytic  evaluations  were  expected  to  identify  and 
define  specific  game  parameters  and  environments,  and  then  obtain  the  subjective  SME  opinions 
regarding  correctness  or  veracity  of  the  games  in  terms  of  existing  doctrinal  policies  or  tactical 
techniques. 


General  Method 

Based  on  prior  analytic  efforts  aimed  at  determining  the  range  and  level  of  possible  skills 
and  knowledge  trained  by  PC  COTS  games  (Morris  &  Tarr,  2002),  two  different  domains  were 
selected  for  analysis.  These  were  dismounted  infantry  and  mounted  armor  operations. 

Rogue  Spear:  Covert  Ops4"1  was  investigated  primarily  because  it  addresses  infantry  squad 
and  fire  team  activities  in  an  urban  environment,  although  other  settings  can  be  played  as  well. 
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The  coordinating  draft  of  FM  90-10-1,  The  Infantryman’s  Guide  to  Modem  Urban  Combat,  was 
used  as  the  basis  for  evaluating  doctrine  in  the  Rogue  Spear:  Covert  Ops'™  PC  game. 

For  the  armor  assessment,  Steel  Beasts4"1  was  selected,  primarily  because  the  mission 
editor  enabled  demonstration  scenario  development.  Steel  Beasts4™  is  an  armor  simulation  that  is 
in  continual  development.  Initial  development  of  the  game  in  1998  apparently  lacked  many  of 
the  features  that  are  considered  today  as  standard.  For  example,  there  were  no  options  to  have  3D 
accelerated  graphics  enabled,  the  sound  quality  was  minimal  at  best,  and  it  had  a  poor  interface. 
The  game  does  have  several  beneficial  features  that  led  to  use  in  this  work.  Primarily,  Steel 
Beasts41”  is  still  being  supported,  with  regular  updates  from  the  developer.  Maintaining  contact 
with  the  developers  was  easy  and  their  responses  were  informative.  External  evaluators,  such  as 
the  on-line  tank-simulation  community,  are  strong  promoters  of  Steel  Beasts4™  and  they  provided 
needed  and  timely  information  (typically  through  internet  game  sites). 

General  Procedures 

In  order  to  attain  detailed  SME  responses  and  to  make  the  evaluation  process  as  easy  to 
accomplish  as  possible  a  focus  group  type  of  evaluation  was  employed,  moderated  and  directed 
toward  attaining  information  on  every  angle  of  the  game’s  potential.  Although  some  of  the 
questions  do  require  different  kinds  of  responses,  the  formats  were  made  as  consistent  and  direct 
as  possible. 

In  order  to  insure  SME  understanding  of  all  game  functions  for  complete  evaluation  of 
their  usefulness  for  military  training,  the  game  flowcharts  were  used.  These  detailed  all  of  the 
game’s  possible  options,  scenarios,  elements,  and  combinations,  as  noted  below.  The  flowcharts 
provided  a  visual  rendering  for  use  in  comparing  the  game  capabilities  to  military  requirements, 
tactics,  and  doctrine. 

The  SME  respondents  were  also  presented  with  the  canned  game  scenarios  which 
demonstrated  functions  such  as  mission  planning,  training  segments,  and  game  options  review. 
Using  these  prepared,  canned  scenarios  and  walkthroughs  allowed  the  researcher  to  present  the 
characteristics  and  features  in  a  game  context  without  requiring  the  SME  to  know  how  to  play 
the  game.  In  this  way,  the  SME  could  determine  just  what  doctrinal  aspects  the  game  supported 
during  play. 


Materials 

In  order  for  SMEs  to  analyze  aspects  of  the  target  games  for  doctrinal  correctness  or 
veracity,  explicit  aspects  of  the  games  had  to  be  clearly  presented  for  reference  to  specific 
military  tasks,  tactics,  and  protocol.  Therefore,  the  games  were  initially  strategically  mapped 
into  detailed  flowcharts  that  clearly  portrayed  game  elements,  options,  and  missions.  These 
flowcharts  were  created  using  Microsoft  VISIO4™.  The  flowcharts  were  developed  by  analyzing 
the  provided  gaming  manuals,  embedded  tutorials,  and  training  scenarios  to  delineate  the 
capabilities  and  note  the  command  sequences  required  for  use.  These  flowcharts  were  also  used 
to  create  canned  game  scenarios  that  mapped  to  military  training,  procedures,  and  protocol.  The 
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game  scenarios  were  developed  under  the  direction  of  in-house  SMEs  working  with  currently 
available  Army  manuals.  For  example,  reviewing  the  game  flowcharts  enabled  the  determination 
of  available  squad  sizes,  briefing  and  debriefing  capabilities,  AI  options,  weapons  options, 
opposing  force  options,  etc.  The  flowcharts  were  used  in  conjunction  with  the  game  scenarios  to 
review  designed  game  components  for  SME  analysis  of  doctrinal  correctness. 

The  hardware  and  software  used  for  the  infantry  and  armor  assessments  consisted  of  a 
portable  projection  system,  projection  screen,  wireless  keyboard/  mouse,  the  game  PC,  PC 
speakers,  installed  Windows  98,  internal  ASUS  V8200  video  card,  laptop,  Rogue  Spear:  Covert 
Ops011  CD,  Steel  Beasts011  armor  CD,  specific  canned  scenario  demonstration  video  (and  back 
up),  camcorder  with  tripod.  An  informed  consent,  demographics  and  expertise  form  (Appendix 
A),  mission  relevant  forms,  game  options  and  elements  forms  (Appendix  E  provides  the  Steel 
Beasts011  materials  as  an  example)  were  used  in  the  assessment.  Minimum  requirements  for  the 
game  are  no  less  than  K6-2m  (or  Intel  Pentium  H011)  400Mhz  with  at  least  128  megabytes  of 
RAM,  and  a  TNT2011  or  comparable  video  card.  Keyboard  and  mouse  are  essential  for  most  of 
these  games,  including  Rogue  Spear:  Covert  Ops11",  and  Steel  Beasts0". 

Experiment  1-  Infantry  Participants 

Two  SMEs  were  used  as  participants  for  the  infantry.  The  first  reported  that  he  had  high 
familiarity  with  infantry  doctrine  (18  years),  high  familiarity  with  general  military  expertise  (23 
years),  was  never  a  platoon  leader  or  company  commander,  but  moderately  familiar  with  Special 
Operations  (12  years)  and  urban  operations  (6  years).  He  never  played  games  and  his  initial 
opinion  is  that  there  are  “no”  benefits  to  using  COTS  PC  games  for  military  training  but  some 
possible  value  added  for  advanced  technologies  in  military  classroom  setting.  The  second 
infantry  SME  reported  high  familiarity  with  infantry  doctrine  (30  years),  high  familiarity  with 
general  military  expertise  (30  years),  high  experience  as  platoon  leader  (7  months),  high 
experience  as  company  commander  (3  years),  and  was  moderately  familiar  with  Special 
Operations  (1  year)  with  no  experience  with  urban  operations.  Additionally,  he  was  highly 
familiar  with  the  combat  training  center  (3  years).  The  second  SME  had  no  opinion  regarding  PC 
games. 


The  coordinating  draft  of  FM  90-10-1,  The  Infantryman's  Guide  to  Modem  Urban 
Combat,  was  used  as  the  basis  for  evaluating  doctrine  in  the  infantry  game.  This  application  was 
combined  with  the  game  maps,  and  the  assessments  were  conducted  via  observation  of 
dialogues. 


Infantry  Evaluation  Procedures 

The  Infantry  SMEs  completed  the  informed  consents,  demographics  questionnaires 
(Appendix  B),  and  then  viewed  the  tactics  oriented  canned  scenarios  followed  by  the  game 
walkthrough  of  planning  phases,  options,  elements,  and  other  potential  missions.  The  SMEs 
addressed  all  potential  doctrinally  effective  and  ineffective  components,  including  potential 
solutions  possible  from  game  alterations.  Finally,  the  SMEs  completed  the  specific 
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questionnaires  and  concluded  their  comments  regarding  the  observations  and  interactions  with 
Rogue  Spear:  Covert  Ops0"  environments,  options,  and  elements. 

Infantry  Results  Summary 

The  SMEs  generated  many  comments  during  the  evaluation  (see  Appendix  B).  The 
comments  have  been  condensed  and  merged  for  a  clear  summary  presentation.  The  biggest 
problems  that  the  SMEs  had  were  with  the  context  of  the  mission,  which  are  doctrinal  violations. 
Primary  among  these  were  comments  about  who,  what,  where,  when,  and  why  in  the 
introduction. 

•  A  9-man  infantry  squad  would  be  used  to  clear  a  5-room  building.  A  platoon  would  be  used 
to  clear  2-3  buildings  (FM  90-10-1,  Jul  2000).  The  PC  game  used  a  4-man  team  (fire  team 
size)  and  the  demo  mission  had  them  clearing  a  series  of  buildings,  which  is  not  realistic. 

•  A  4-man  fire  team  cannot  support  itself.  Once  a  team  clears  an  area  some  element  must 
remain  in  place  or  another  must  relieve  it  in  place. 

•  It  is  valid  to  use  notional  support  elements,  but  their  use  and  role  must  be  addressed  during 
the  introduction.  In  addition,  there  must  be  a  clear  order  [Rules  of  Engagement  (ROE)/Rules 
of  Interaction  (ROI)]  regarding  hostages. 

•  It  is  normal  to  use  “weapons  tight”  or  ROE  -restrictive  (return  fire  only)  when  encounters 
with  hostages  are  expected. 

•  The  focus  for  a  particular  scenario  snippet  from  the  game  must  be  narrow  and  clearly 
defined.  For  example,  some  of  the  movement  tactics  (e.g.,  entry  techniques)  and  character 
movements  were  incorrect  (e.g..  weapon  positions). 

SMEs  also  objected  to  aspects  in  the  kit  selection,  for  example  a  full  metal  jacket  round  is  always 
used  because  a  hollow  point  is  against  the  Geneva  Convention. 

The  SMEs  concluded  that  the  game  could  be  used  for  training  in  spite  of  the  doctrinal 
problems,  as  long  as  trainers  attended  to  those  negative  aspects  present  in  the  training  vignette. 
However,  additional  features  would  be  needed  to  enhance  training.  For  example,  there  is  a  need 
to  capture  (hear)  dialogue  between  squad  members,  especially  during  the  After  Action  Replay. 
Also,  tactical  movement  in  the  game  is  relatively  realistic  if  it  is  made  clear  that  participants 
have  an  additional  support  team  or  platoon  in  an  overwatch  capacity  (see  comments,  above). 
Overall,  the  infantry  SMEs  especially  liked  the  opportunities  to  practice  aspects  of  room  and 
building  clearance,  the  ability  to  form  the  "stack"  at  the  entrance  to  a  room  and  moving 
systematically  throughout  a  building.  The  consensus  was  that  Rogue  Spear:  Covert  Opstm 
provides  users  with  a  realistic  simulation  of  many  facets  of  covert  operation  missions.  Rogue 
Spear:  Covert  Opstm  also  offers  an  excellent  multi-player  cooperative  capability  to  exercise  team 
skills  among  individuals.  Both  SMEs  were  open  to  using  the  game  in  an  early  training  stage  for 
new  troops. 

Experiment  2- Armor 

For  the  Armor  review,  challenges  in  scheduling  and  new  policy  on  use  of  soldiers  for 
data  collection  resulted  in  our  not  having  access  to  personnel  at  Ft.  Knox;  one  issue  was  the 
general  heightened  security  alert  as  a  result  of  the  incidents  on  September  11, 2001.  Therefore, 
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local  SMEs  were  engaged.  These  were  retired  military  armor  officers  who  were  currently 
employed  by  the  Army  for  their  expertise  on  Armor  operations.  Three  armor  SMEs  took  part  in 
the  armor  review.  In  addition  to  a  strong  armor  background,  each  is  a  retired  Army  colonel  (0-6) 
with  significant  military  experience.  Each  also  has  active  duty  training  credentials  and 
credentials  based  on  their  current  position  as  government  contractors  working  on  training 
simulation  issues  for  Project  Manager  for  Combined  Arms  Tactical  Trainers  (PM  CATT)  and 
Project  Manager  for  Warfighters’  Simulation.  One  SME  had  served  as  commander  of  the 
opposing  forces  at  the  National  Training  Center  —  undoubtedly  the  U.S.  Army’s  most  rigorous 
training  environment  for  armored  and  mechanized  infantry  forces  —  and  later  as  Commander  of 
an  Armored  Training  Brigade  at  Ft.  Knox.  Another  served  as  the  Training  and  Doctrine 
Command  System  Manager  for  the  Combined  Arms  Tactical  Trainer  and  assisted  in  the  detailed 
collective  task  analysis  of  armor  crew  tasks  as  part  of  the  Training  Exercise  Development 
System  study.  The  Training  Exercise  Development  System  is  the  training  support  package  for 
the  Close  Combat  Tactical  Trainer  (CCTT)  developed  by  PM  CATT  to  help  unit  commanders 
efficiently  allocate  collective  training  tasks  to  structured  CCTT  training  exercises. 

The  discussion  used  revised  mission  scenarios  developed  specifically  by  using  Steel 
Beasts*™'  mission  editor  to  incorporate  Platoon  Reaction  Exercises  from  the  Tank  Tactical  Tables 
in  FM  17-12-1-2  (Tank  Gunnery).  Specifically,  scenarios  for  missions  1-2  (Engage  Multiple 
Targets)  and  1-4  (React  to  Ambush)  have  been  incorporated  into  the  game.  The  incorporation  of 
tactical  table  scenarios  had  the  potential  to  offer  some  doctrinal  consistency  in  training  value. 
Since  the  Tank  Tactical  Tables  were  applied  to  the  evaluations  for  armor  SMEs,  there  were  no 
forms  and  the  assessments  were  conducted  via  observation  of  dialogues. 

Procedure — Armor  Analysis 

The  SMEs  completed  informed  consents.  Then  the  expert  gamers  provided  a  basic 
overview  of  the  Steel  Beasts*™  game  functionality  and  capabilities.  The  evaluation  began  with  a 
brief  discussion  of  infantry  findings  and  the  rationale  for  selecting  Steel  Beasts*™.  Gamers  also 
explained  the  capability  of  the  mission  editor,  and  the  SMEs  were  told  that  it  was  a  primary 
reason  Steel  Beasts*™  was  selected,  that  the  game  is  continuously  being  upgraded  and  developers 
will  release  an  M1A2  version  in  the  spring.  The  Armor  demonstration  focused  on  the  gunnery 
portion  of  the  simulation  because  a  consensus  agreed  that  no  simulation  currently  available  to  the 
public  could  accurately  reflect  the  driving  tasks  and  activities  of  the  tank.  The  fact  that  the 
simulations  were  not  developed  with  military  application  in  mind  was  acknowledged.  It  was  also 
made  clear  that  realism  has  been  sacrificed  for  playability  or  “fun  factor.” 

An  overview  of  the  Steel  Beasts*™  game,  missions,  and  capabilities  (including  the  mission 
editor)  was  briefed  and  discussed  with  the  SMEs.  Introductory  comments  were  made  regarding 
other  uses  of  PC  off-the-shelf  games  in  training  environments  and  the  objective  of  this  project. 
The  introduction  stressed  the  goal  of  identifying  aspects  or  elements  of  the  game  that  might  have 
doctrinal  utility,  or  contribute  to  doctrinal  ineffectiveness  in  individual  or  collective  training. 
Then  the  Tactical  Table  missions  incorporated  in  the  game  were  demonstrated.  Two  expert 
gamers  served  as  a  tank  section,  as  the  TC  and  gunner  of  the  same  tank  or  as  opposing  tankers 
while  key  tactics  were  demonstrated.  This  assessment  also  consisted  of  a  strategic  mix  of  game 
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research,  game  diagnosis  and  mapping,  design  assessment,  evaluation  of  potentially  applicable 
tactics,  and  examination  of  relevant  doctrine  references.  Reaction  exercises  from  the  Tank 
Tactical  Tables  in  FM  17-12-1-2  were  included  in  the  developed  demonstration. 

Armor  Results  Summary 

A  feature  of  Steel  Beasts011  that  was  considered  beneficial  by  the  SMEs  was  its  mission 
editor,  which  allows  the  user  to  easily  and  quickly  create  a  mission.  The  interface  for  this 
function  was  determined  to  be  intuitive,  and  allows  for  replication  of  current  military  training 
procedures.  The  unit  choices  were  considered  well  balanced  because  users  are  given  the 
opportunity  to  select  anything  from  an  M1A1  to  a  Russian  T-72  or  an  M2A2  Bradley.  The 
mission  editor  is  unique  and  it  provides  Steel  Beasts0"  with  its  biggest  advantage  because  the 
user  can  manipulate  the  game  to  fit  the  practice  and  training  session  needs. 

During  the  demonstration  of  Steel  Beasts0",  SMEs  noted  that  the  tracking  and  lasing 
(sighting  with  a  laser)  seemed  jerky.  The  deficiency  was  traced  back  to  the  joystick  being  used, 
the  MS  Sidewinder  Force  Feedback.  Also  noted  as  a  problem  was  the  manual  switch  between 
HEAT  round  targeting  and  sabot  round  targeting  that  Steel  Beasts0"  required.  The  SMEs  weren’t 
impressed  with  the  targeting  reticule  Steel  Beasts00  provided,  nor  with  the  fact  that  Steel  Beasts0" 
offered  only  the  Ml  Al,  and  not  the  more  modem  Ml  A2  or  M1A2  Special  Equipment  Package. 
Furthermore,  the  SMEs  noted  that  during  the  gunnery  exercise,  after  a  tank  had  been  hit  and 
disabled,  it  shouldn’t  disappear  but  should  smoke  and  stop  moving. 

Although  Steel  Beasts0"  does  not  include  the  modem  Ml  A2,  it  was  concluded  that  it  was 
a  good  simulation  when  its  gunnery  aspects  are  considered.  During  analysis,  a  strong  user  base 
was  noted,  and  the  developers  were  accessible  and  responsive.  Multiplayer  capability  over  the 
Internet  made  battles  easy  to  arrange  and  play.  In  addition,  these  features  contribute  to 
cooperative  play.  It  was  determined  from  the  Steel  Beasts0"  environment  demonstration 
preparation  that  the  multiplayer  function  allowed  more  than  20  users  in  the  same  mission 
simultaneously.  Two  users  can  be  in  the  same  tank  at  the  same  time,  one  as  the  Tank 
Commander  and  one  as  the  gunner.  Under  optimal  conditions,  eight  people  would  play  a  mission 
as  one  tank  platoon  of  four  tanks,  with  a  separate  Tank  Commander  and  Gunner  for  each  tank. 
During  testing,  however,  we  were  unable  to  have  a  full  platoon.  It  was  during  this  testing  phase 
that  the  computer’s  AI  inadequacies  were  noted  (in  comparison  to  that  of  a  normal  human).  (It 
was  also  noted  by  the  gaming  demonstrators  that  according  to  doctrinal  guidelines  and  the 
context  of  the  game  under  study,  the  number  of  players  required  to  portray  demonstrable 
missions  varied.  During  train  up  of  initial  game-crew  demonstrators,  several  issues  tend  to 
complicate  the  evaluation  potential.  For  example,  after  a  group  of  two  or  more  individuals  had 
been  practicing  a  particular  game  together,  adding  an  extra  player  disrupted  the  mission  flow. 
Overall,  teaming  became  more  difficult  as  those  early  players  had  gained  a  situational  awareness 
that  new  players  didn't  have.) 

Steel  Beasts0"  has  its  drawbacks.  First,  the  options  are  very  limited,  as  mentioned  above, 
because  only  the  Ml  Al  and  the  German  Leopard  2  tanks  are  available.  Other  users  have  made 
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modifications  to  these  tanks  in  order  to  reflect  other  well-known  models  but  what  they  have 
affected  is  little  more  than  a  different  paint  job  or  “skin.” 

The  evaluation  of  Steel  Beasts*"1  resulted  in  the  SMEs  deciding  that  despite  the  many 
strong  elements  in  the  game,  it  had  facets  that  could  potentially  result  in  negative  training  or 
cause  confusion  on  the  part  of  soldier  students.  The  greatest  disadvantage  of  the  game  was  the 
lack  of  the  A1  armor  system  and  the  game’s  limited  mission  planning  capability,  unless  the 
mission  editor  was  employed.  The  SMEs  had  been  informed  that  the  same  development  team  is 
working  on  Steel  Beasts  IItm,  which  would  include  the  Ml  A2,  and  that  inclusion  was  perceived 
as  necessary  for  doctrinal  effectiveness  in  training  according  to  all  SME  evaluations.  Specific 
comments  from  the  SME  discussions  are  provided  in  Appendix  D. 

Discussion 

The  objective  for  this  project  was  to  examine  the  doctrinal  correctness  of  at  least  two 
selected  COTS  PC  games  in  terms  of  Army  Standards,  to  further  establish  the  actual  training 
utility  for  meeting  training  requirements,  as  a  follow  on  to  the  initial  study.  A  usable  and  easy  to 
implement  process  was  attempted,  consisting  of  a  straight  forward  analysis  of  game 
characteristics  and  development  of  an  exemplar  scenario.  This  information  was  used  in  focused 
group  discussions  with  available  SMEs  to  investigate  doctrinal  correctness.  This  approach 
should  be  considered  as  an  option  that  could  prove  to  be  extremely  beneficial  to  military 
agencies  that  are  considering  integrating  COTS  PC-based  synthetic  environments  into  training 
programs.  The  key  in  the  approach  is  describing  and  demonstrating  game  characteristics,  and 
producing  a  relevant  demonstration  scenario  for  SME  evaluation. 

The  difficulty  of  analytically  assessing  actual  military  training  needs  might  also  affect 
future  analysis.  As  contractors  and  ex-military  personnel,  obtaining  up-to-date  documentation 
specific  to  the  potential  of  the  game  under  study  was  difficult.  In  order  to  understand  the  games 
and  their  usefulness  for  military  training,  the  games  were  mapped  into  flowcharts  depicting 
characteristics,  features,  and  options.  In  order  to  interpret  these  maps,  equally  detailed  military 
training  needs  were  necessary.  Early  in  the  analytical  process,  it  was  noticed  by  our  resident 
SMEs  that  the  tactics  used  for  these  scenario  demonstrations  were  somewhat  outdated,  both  in 
the  games  and  in  terms  of  the  dates  of  the  available  documents.  Also,  the  in-house  SMEs  were 
aware  of  several  new  techniques  for  urban  terrain  and  special  operations  use  of  smaller  teams 
that  were  not  reflected  in  the  games  or  in  the  available  Army  manuals. 

The  consensus  of  the  SMEs,  both  in-house  and  Government  provided,  was  that  the 
Infantry  Game,  Rogue  Spear:  Covert  Ops“"  could  be  used  for  relatively  doctrinally  correct 
tactical  movement,  tactical  scenario,  and  room  clearing,  although  other  elements  had  some 
drawbacks.  As  to  the  armor  game,  although  the  current  edition  of  Steel  Beasts*”1  does  not  include 
the  modem  M1A2,  it  was  considered  a  good  simulation  when  its  gunnery  aspects  were 
considered  solely.  However,  the  evaluators  believed  that  there  was  a  sufficient  number  of  small 
doctrinal  errors  in  Steel  Beasts*"1  that  the  overall  game  was  not  useful.  They  indicated  that 
establishing  a  tie  in  to  the  collective  tasks  from  Automated  Systems  Approach  to  Training 
database  with  specific  aspects  of  the  game  could  be  a  way  to  overcome  the  problem. 
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Conclusions 


The  potential  usefulness  of  COTS  PC  games  needs  to  be  studied  further  with  actual 
clinical  trials  of  soldiers  being  used  and  assessed  against  actual  training  requirements.  The  game 
mapping  process  developed  in  this  effort  is  one  technique  that  could  streamline  such  efforts  and 
add  to  the  effectiveness  in  identifying  what  PC  COTS  games  elements  are  viable  candidates  for 
inclusion  in  training.  Clearly  elements  of  the  studied  games  are  useful,  especially  when  organic 
mission  editors  can  be  used  to  tailor  the  relevant  portion  of  the  games  to  satisfy  real  training 
requirements.  Feedback  from  the  selected  SMEs  indicated  that  with  mission  editors  and  a  “guide 
for  instructors”  certain  aspects  of  the  games  would  be  useful  and  might  provide  more  engaging 
instructional  approaches  than  are  currently  used. 

The  authors  do  not  advocate  using  a  gaming  environment  as  is.  Instead,  instructors 
should  integrate  strategically  evaluated  and  designed  game  features  and  components  within  an 
instructional  design  approach  for  specific  training  instances  such  as  classroom  enhancement  aids, 
low  cost  complimentary  practice  missions,  and  demonstration  training.  The  processes  used  in 
this  and  previous  related  efforts  (Morris  &  Tarr,  2002)  are  initial  and  exploratory,  addressing  the 
wide  range  of  evaluations  that  are  necessary  before  incorporating  any  product  into  a  training 
program.  Given  the  visual  and  game  entertainment  orientation  of  our  country,  PC  games  can 
contribute  as  part  of  a  total  training  system. 
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Appendix  A 

Experience  and  Demographics  Form 


A-l 


Participant  Information 

1.  Age? 

2.  Gender? 

3.  Affiliation  (check  all  that  apply)  UCF _  1ST _  Military _ 

4.  Title _ 

5.  On  average,  how  often  have  you  played  the  following  types  of  PC  or  video  games  (on  any  console)  in  the 
past  year? 

Shooter 

12  3  4 

Never  1-10  hrs/week  11-20  hrs/week  over  20  hrs/week 

Please  list  any  games  you  have  played  within  this  category _ 

Strategy 

12  3  4 

Never  1-10  hrs/week  11-20  hrs/week  over  20  hrs/week 

Please  list  any  games  you  have  played  within  this  category _ 

Simulation 

12  3  4 

Never  1-10  hrs/week  11-20  hrs/week  over  20  hrs/week 

Please  list  any  games  you  have  played  within  this  category _ _ 

Real-Time  Action 

12  3  4 

Never  1-10  hrs/week  11-20  hrs/week  over  20  hrs/week 

Please  list  any  games  you  have  played  within  this  category  _ 

6.  Are  you  a  Psychology  major  who  has  taken  a  Basic  Learning  course,  Cognitive  Psychology  course  or 
equivalent? 

Yes _  No _ 

7. Are  you  familiar  with  analysis  of  military  training? 

Yes _  No _ 
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Appendix  B: 


Mission  Relevant  Charts 
for  Rogue  Spear: 
Covert  Ops 


B-l 


DPQ  -  RUN  3  Planning  phase-  Covert  Operations  Mission  CD 


Doctrinal  Correctness 

Check  off  box  that  applies 

Army 
Standard/ 
or  YES 

LO 

lo 

LO 

to 

LO 

LO 
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CM 

CM 

CM 

Not  at 
all 

- 

- 

- 

- 

- 

- 

v— 

- 

- 

x— 

1 

- 

- 

r- 

Training  Applications 

Rank  YES=1  Maybe=2  No=  3  and  comment  as  much  as  possible 

Sustaining/Practice 

Skills 

1 

Train  Ups,  Initial 

Classroom 

Enhancement 

Multiplayer 

Create  game 

Server  only  control 
buttons 

-Lock  teams 

-  Kit  Restrictions 

-Eject  Player 

-  User  Password 

-  Lock  Server 

-  Main  menu 

Cooperative  Team 

Map  =  (i.e.  Kosovo) 

Difficulty  level  = 

Primary  Weapon  = 
M16-A2 

Secondary  Weapon  = 
Beretta  92FS  9mm 

Ammunition  type  = 

Full  Metal  Jacket 
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DPQ  -  RUN  3  Planning  phase-  Covert  Operations  Mission  CD 


Doctrinal  Correctness 

Check  off  box  that  applies 

Army 
Standard/ 
or  YES 

LO 

ID 

LO 

LO 

LO 

LO 

LO 

LO 

LO 

to 
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LO 

nt 
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- 

1 

Training  Applications 

Rank  YES=1  Maybe=2  No=  3  and  comment  as  much  as  possible 

Sustaining/Practice 

Skills 

Train  Ups,  Initial 

| 

] 

Classroom 

Enhancement 

i 

Equipment  =  M61 

Frag  Grenade 

Uniforms  =  Wood 
Medium 

Join  Game 

Game  options 

-  Auto  targeting 

-  blood 

-  record  mission 

-  unlimited  practice 

Mouse  settings 
-  mouse  look,  invert, 
sensitivity 

Predefined  interface 
settings 

-  Preferred  map  zoom 

-  Preferred  rate  of  fire 

-  Preferred  Action 
display 

DPQ  -  RUN  3  Planning  phase-  Covert  Operations  Mission  CD 


Doctrinal  Correctness 

Check  off  box  that  applies 

Army 
Standard/ 
or  YES 
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in 
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CM 

Not  at 
all 

- 

T- 

- 

- 
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- 
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- 

- 

- 

- 

- 

- 

Training  Applications 

Rank  YES=1  Maybe=2  No=  3  and  comment  as  much  as  possible 

Sustaining/Practice 

Skills 

Train  Ups,  Initial 

Classroom 

Enhancement 

Sounds 

-  Master 

-  Ambiant 

-  Gunshot 

-  Footstep 

-  Radio 

Chatter 

-  Shell  sound 

-  Action  music 

-  CD  music 

-  Use  maximal 

Graphics 

-  Force  software 

-  Video  resolution 
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Doctrinal  Correctness 

Check  off  box  that  applies 

Army 
Standard/ 
or  YES 

LO 

1 
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1 
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in 
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■ 

1 

■ 
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CM 
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- 
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- 

- 

- 

- 

- 

Training  Applications 

Rank  YES=1  Maybe=2  No=  3  and  comment  as  much  as  possible 

Sustaining/Practice 

Skills 

Train  Ups,  Initial 

1 

Classroom 

Enhancement 

-  Detail  level 

-  Use  Action  Gamma 

-  Show  tooltips 

Advanced  Options 

-  Show  sky,  Fog, 

Weather  Effects 

-  LOD  skip 

|  -  MIP  skip 

-  Dynamic  lighting 

-Character  shadows 

-  Projected  textures 

-  Show  weapons  after 
death 

-  Bullet  holes 

-Texture  filtering 

|  MP  Game 

-Name 

-Chan 

-  Voice  pitch 

B-5 


DPQ  -  RUN  3  Planning  phase-  Covert  Operations  Mission  CD 


Doctrinal  Correctness 

Check  off  box  that  applies 

Army 
Standard/ 
or  YES 
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Training  Applications 

Rank  YES=l  Maybe=2  No=  3  and  comment  as  much  as  possible 

Sustaining/Practice 

Skills 

1 

1 

1 

1 

Train  Ups,  Initial 

1 

1 

1 
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1 

Classroom 

Enhancement 

-behind  firewall 

-Use  arm  patch 

-  Connection 

-  Gender 

-  Network  address 

MP  Server 

-Server  name 

-Message  of  the  day 

-  Max  #  of  players 

-  Blue  Team  size 

1  -Gold  team  size 

-  join  port 

-  Info  port 

-  Announce  port 

1  -Use  password 

-  Allow  arm  patches 

-  Force  Fog 

-  Force  weather  effects 
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7.)  ..Define  Battlefield  environment,  describe  battlefield  effects,  evaluate  the  enemy, 

determine  enemy  COA. _ 

7a.)  ..Issue  warning  order  (follow  5  para  OPORD  format)? 
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aspects  of  the  game.  AAR  should  marry  audio  with  stealth  view. 
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Appendix  C-  Infantry  Evaluation  Detailed  Results 


Game  options, 
capabilities  & 
events 
addressed 

Scores  represent  (1 )  =  no  doctrinal  value  to  (5)  =  Yes,  army 
standard. 

With  noted  issues,  comments,  and  solutions 

Mission  briefing 

(5)  Has  utility  to  use  as  is. 

(3) 

Break  it  down  to  a  team  leader  level 

Mission  intel 
categories  / 
subjects 

(1)  Too  game-ish 
(5)  Good 

If  tailored  to  a  TM  or  squad  mission  may  have  utility 

Make  less  game  like 

Planning  phase 

(1 )  Not  realistic 

Make  less  game  like 

If  made  like  weapons  selection  adding  a  menu  and  secondary 
systems  (i.e.,  rafeal  or  bunker  buster,  slope  changes,  AF-8  etc.) 

Roster  screen 

(1)  Not  necessary 

(1)  Delete,  it  is  not  realistic 

Kit  selection 

(5)  Has  Frags/Flashbang  ml  6  vs  m4 
(5) 

(4) 

Make  less  game  like 

Like  mission  tailoring  capability  need  to  expand  listing. 

Less  focus  on  exotic  offer  more  changes 

Primary  weapons 

(5)  Good  round  count  forces  player  to  change  mags 

(2)  not  much  choice  in  TU&E  units 

(5) 

Make  less  game  like 

Secondary 

weapons 

(1)  Privates  don’t  carry  secondary  weapons 
(4)  Some  utility 

Constrain  any  use  of  secondary  weapons 

Less  focus  on  expotic  systems  more  on  various  systems  to 
support  mission 

Ammunition  type 

(1)  Not  realistic 
(1) 

Turn  off  option 

Equipment 

(1)  Not  realistic 
(5) 

Turn  off  option 

More  choices  of  equipment  to  support  mission 
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Scores  represent  (1)  =  no  doctrinal  value  to  (5)  =  Yes,  army 
standard: 

With  noted  issues,  comments,  and  solutions 


Game  options, 
capabilities  & 
events 
addressed 


Uniforms 


Team  selection 
Blue, red, green, 
and  gold 


Mission  plan 


Orders 


Plotting 
movement 
-  map  controls 
Rules  of 
engagement 


Speed 


Special  actions 


Ability  to  exercise 


Ability  to  execute 

Batallian 

Commanders 

intent _ 

Specified  and 
applied  missions 


(1)  Never  a  choice 

(4) 

Turn  off  option 


(1 )  Not  necessary 

(2) 

Need  only  good  and  bad 


(1 )  Possible  but  not  shown  here 

(5)  Supports  good  planning  (i.e.,  room  clearing) 

(5)  Excellent  rehearsal  tool 

Too  many  buildings  a  platoon  can  clear  2-3. 


(1 )  Not  realistic,  too  game  like 
(2 


(1 )  Not  realistic,  too  game  like 

(2) 


(1 )  too  game  like 
T 


(1)  will  only  be  used  as  demo 
T 


(1 )  Not  realistic 
1 


(5)  Valuable  if  done  to  doctrine 
(5)  Use  of  map  is  superb 


(5)  Valuable  if  it  is  a  realistic  intent 
(1 )  not  appropriate  at  squad  level 


(5)  if  they  are  covered  in  the  AAR 
(1) 

Need  AAR 


8  step  troop 

(1) 

leading  process 

(5)  has  Capability 

demonstrable? 

(D 

ability  to  apply  or 

(5) 

practice 

make  less  game  like 

Add  a  laminated  troop  leading  process  check  list  to  the  job  aides 
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Game  options, 
capabilities  & 
events 
addressed 

Scores  represent  (1 )  =  no  doctrinal  value  to  (5)  =  Yes,  army 
standard. 

With  noted  issues,  comments,  and  solutions 

Kinds  of  logistics 

potentially 

demonstrated 

(D 

(2) 

Needs  to  be  kept  short  with  AAR’s  to  instill  learning 

Option  (auto 
targeting) 

(1)  Needs  to  replicate  PAQ  4c,  PEQ2a,  TWS  MRD,  PVS-14,  CCO 
(1) 

If  could  be  made  to  replicate  CCO 

Disengage,  turn  it  off 

Option:  blood 

(1 )  Not  realistic 

(3)  Good  for  psychological  impact 

Could  not  be  made  to  be  realistic 

Option:  recording 
mission 

(5) 

(5) 

Only  in  observer  view/stealth 

Needs  to  be  added  into  collective  training 

Add  audio 

Interface: 
preferred  map 
zoom 

(5)  Very  good  to  demo  L.W.  capabilities 
(5) 

map  zoom  only 

Enlarge  for  AAR  purposes.  Make  sure  player  cant  see  that  you 
are  recording 

Interface: 

Preferred  rate  of 
fire 

(1 )  not  realistic,  menu  selected 
(5)  good  tool 

Needs  to  be  kept  adjustable  during  mission 

Interface: 
preferred  action 

display _ 

Sounds:  gunshot 


Sounds:  footstep 

Sounds  radio 

chatter _ 

Sounds:  shell 
sound 


(5)  Good  support  first  person  view 

(5)  keep  variable  display.  Good  for  AAR  purposes 

(1)  not  loud  enough 
(5)  KEEP 

will  never  be  loud  enough _ 

(1)  not  realistic 

(5)  KEEP  _ 

(1 )  not  realistic 

(1)  meaningless  chatter  is  useless _ 

(1 )  not  realistic 
(3)  marginal  value 
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Game  options,  Scores  represent  (1 )  =  no  doctrinal  value  to  (5)  =  Yes,  army 
capabilities  &  standard. 

events  With  noted  issues,  comments,  and  solutions 

addressed _ 

Sounds  action  (1 )  not  realistic 

music  (1) 

_ Delete _ 

Graphics:  (1) 

resolution  &  (1 )  N/A 

detail _ _ _ 

Graphics:  (1)N/A 

Tooltips _ 

Sky,  fog,  (2)  not  realistic 

lightening,  (5)  rain  and  fog  are  ok  and  needed  for  limited  visibility  training 

weather _ Lightening  is  a  waste  of  time _ 

Shadows:  (4)  Good,  can  shoot  through  doors  and  windows,  however,  walls 

character,  can’t  be  shot  through 

buildings,  and  (4)Need  to  be  able  to  shoot  through  walls 

projected  Would  be  nice  to  be  able  to  adjust  time  of  day  as  opposed  to 

textures _ game  dictating _ 

Number  of  (4)  Can  display  the  necessary  4  man  stack. 

players,  (5)  can  use  work  arounds  to  replicate  squad  and  X  number  of 

multiplayer  OPFOR 

capabilities  and  Need  to  be  able  to  make  personnel  varied  so  that  one  can  tell 

teaming  abilities  others  to  do  something _ 

Targeting  reticule  (1)  movement 
and  ridicule  (1)  stance  change 

modifiers  (1)  fatigue  level 

(1)  health  level 
(1)  firing 

(4)  taking  hits,  good  because  can  make  player  limp 

(5)  flashbangs,  realistic 

(5) 

_ delete  fatigue  and  health  variables _ 

Action  bar  (5)  map  window,  great  L.W.  display 

(1)  weapons  display 
(1)  active  team  display 
(1)  all  teams  display 
(1) 

_ Delete  all _ 

Control  (5)  Good 

(movement, 

weapons,  doors)  _ 
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Game  options,  Scores  represent  (1)  =  no  doctrinal  value  to  (5)  =  Yes,  army 

capabilities  &  standard. 

events  With  noted  issues,  comments,  and  solutions 

addressed _ _ 

Replay  AAR  (5)  Great  capability 

(4) 

_ Must  have  someone  to  conduct  the  AAR _ 

Cover  and  (3)  Could  possibly  be  done  if  solid  furniture  were  provided, 

concealment  Not  able  to  assume  prone  position 

(5)  the  multi-floor  diagrams  are  a  plus  for  rehearsals 

_ Need  furniture  and  ability  to  assume  prone  position _ 

Team  security  (4)  supports  it  and  can  be  done 

-  Flank,  assume  (5) 
defensive 
postures,  cover  / 
counter  attack 

routes _ 

Mutual  support  (5)  Can  be  done  in  the  game 

_ J5) _ 

Fields  of  fire  (4)  Can  be  done  in  game  if  provided  proper  instruction 

_ M _ 7 _ 

Leadership  skills  (4)  Could  be  done  in  the  game,  great  for  leadership  on 
contributing  kit  selection 
(5) 

_ Needs  to  be  slower  than  shown _ 

Door  breaching  (4)  Could  be  done  and  teach  blast  radis  etc. 

_ J5) _ 

Assault  (5)  Could  be  done  in  game 

procedures _ need  stacking  capabilities _ 

Covering  fire  (5)  could  be  taught  in  game  if  the  game  were  slower.  Game 
moved  too  fast 

_ Slow  the  game  down _ 

Covered  routes  (5)  Could  be  done  with  ground  view  of  street,  alley  etc. _ 

Room  clearing  (5)  Could  be  taught  with  overhead  view,  then  practicing  in  the 

procedures  game 

_ Need  a  way  to  grade  the  game  after  played _ 

Communication  Supports  good  interaction  and  planning  if  inter  team  sound 
systems  employed 

_ Need  outside  noise  distraction  and  team  radios _ 

Re-supply  and  (5)  magazine  changes  could  be  taught  here 

re-arm  Round  count  is  a  major  plus.  Changing  magazines  is  very 

realistic. 
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Game  options,  Scores  represent  (1 )  =  no  doctrinal  value  to  (5)  =  Yes,  army 
capabilities  &  standard. 

events  addressed  With  noted  issues,  comments,  and  solutions 


AAR  or  mission 
replay 

(1)  not  a  function  of  game 

Game  does  not  allow  for  replay  in  multi-player  mode 

Need  overhead  view  similar  to  the  floor  diagram  in  mission 
rehearsal 

(4)  if  accommodations  made  so  to  display  it 

User  must  record  mission  using  alternative  output  replay  devices 
(i.e.  VCR) 

Need  to  record  audio  in  alternative  fashion  as  well 

Game  fidelity 

(1)  grenade  throwing  not  realistic  functioning 

Psychological 

issues 

(1)  game  too  difficult,  promotes  negative  learning 
can  not  simulate  blood  and  death 

players  can  eventually  win  missions  only  by  memorizing  what  the 
enemy  Al  will  do 

Weapons 

orientation 

(1)  could  be  taught  but  cannot  be  controlled  with  a  pc 
(1)  grenades/flashbangs  cannot  train  for  airburst 

Overall  game 
potential  to  allow 
doctrinally  correct 
procedures 

(4)  has  potential  but  not  as  demonstrated  for  this  event 

practice  for  small  aspects  (not  whole  town)  but  very  specific,  narrow 

and  defined  with  narrow  scope  and  narrow  OPFOR 

Capability  for 
good  classroom 
enhancement  (i.e. 
demo’s) 

(5)  Definitely  has  use:  demonstration  on  large  screen  with  all  pro’s 
and  con’s  pointed  out 

Potential  uses:  Room  clearing  can  build  into  scenarios  covering 
other  troops.  Movement  program  has  flaws  and  plusses.  Tactical 
scenario  training 

Capabilities  for 
initial  train  ups 

Has  some  potential  for  initial  train  ups 

RC  movement 

need  a  better  demonstration 

Skills  practice 
and  sustaining 
skills 

(2)  Once  the  basics  are  learned,  a  game  is  not  the  place  to  sustain 
proficiency 

RC 

Interoperation,  independent  thought  applied  to  tactic 

To  get  roles  down 

Use  map  views.  Good  here 
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_ Appendix  G:  Armor  results _ 

ARMOR  responses  and  potential  solutions 

•  Must  determine  what  we  are  trying  to  teach  -  Gunnery,  command  and  control, 
individual  tasks,  collective  tasks? 

•  SB  plays  a  four  tank  platoon  (Ml  A1  s)  with  options  for  a  TC  and  gunner  in  the 
same  tank,  or  TCs,  or  gunners  in  two  different  tanks.  The  loader  is  not  played 
and  the  TC  performs  driver  functions  which  is  not  doctrinally  correct. 

•  A  determination  should  be  made  on  whether  the  game  is  better  suited  as  a 
teaching  aid,  or  trainer,  for  tank  gunnery  or  combined  arms  command  and 
control. 

•  The  game  does  not  allow  the  TC  to  override  the  gunner  in  target  engagements 
-  considered  a  serious  drawback. 


•  Better  to  use  Mission  T raining  Plans  (MTPs)  that  provide  a  complete  set  of 
individual  and  crew  tasks  rather  that  the  platoon  reaction  exercises  which  are 
more  general  in  nature. 

•  The  game  must  allow  the  tanks  to  fire  on  the  move.  Tanks  under  computer 
control  stop  before  they  engage  a  target. 

•  Players  must  tell  the  game  to  allow  tanks  to  fire  while  on  the  move.  This  is  a 
serious  problem  for  training  purposes. 

•  Users  are  not  permitted  to  view  and  modify  algorithms  for  OPFOR.  In  order  to 
provide  realism  the  operators  (trainers)  must  be  able  to  view  and  modify  the 
algorithms  used  for  both  OPFOR  and  BLUFOR.  This  would  permit  target 
engagements  as  soon  as  tanks  are  within  line-of-sight. 

•  C2  is  difficult  to  incorporate  into  the  game  due  to  the  fact  that  the  Platoon 
Leader,  as  a  TC,  is  basically  the  tank  driver  in  the  game. 

•  While  the  TC  does  instruct  the  driver,  the  Pit  Ldr  must  also  command  his  tank 
and  coordinate  the  activities  of  the  other  three  tanks  in  his  platoon. 

•  The  players  mentioned  that  they  had  never  defeated  the  OPFOR  in  the  game. 
That  in  itself  is  negative  training. 
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•  A  better  use  of  the  game  would  be  to  set  up  a  gunnery  lane  -  a  firing  range 
whereby  one  tank  engages  different  types  of  targets  -  tanks,  trucks,  troops  in 
the  open,  etc. 

•  An  approach  similar  to  Tank  Table  X,  if  still  in  use,  would  be  beneficial. 

•  The  game  would  be  a  precursor  to  training  in  a  UCOFT  or  TWGSS  and  would 
begin  by  engaging  stationary  targets  from  the  halt.  The  emphasis  would  be  on 
target  acquisition  and  the  coordination  between  TC  and  gunner  -  proper 
commands,  proper  procedures  -  target  tracking,  ammunition  selection,  etc  to 
ensure  a  successful  engagement. 

•  Difficulty  could  be  added  as  proficiency  increased  -  e.g.  firing  on  the  move 
against  moving  targets,  and  timed  exercises  whereby  targets  engage  your 
tank  as  line-of-sight  is  established. 

•  During  the  play  of  the  game  there  was  little  or  no  dialogue  between  TC  and 
gunner  players.  It  was  played  as  two  gamers  trying  to  engage  as  many  targets 
as  possible  rather  than  following  proper  target  engagement  procedures  which 
is  not  enforcing  doctrine. 

•  The  computer-controlled  tanks  automatically  considers  dismounted  soldiers  a 
greater  threat  than  opposing  tanks  and  tend  to  slew  off  tanks  and  on  to 
dismounted  soldiers  for  initial  engagements  which  is  in  opposition  to  military 
tactic  and  doctrine. 

•  The  game  gives  no  credit  for  dismounted  soldier  engagements  or  kills. 

•  There  was  a  lack  of  team  integration.  Tank  gunnery  is  a  team  effort  and  player 
coordination  must  be  a  part  of  the  game.  (Flying  a  plane,  on  the  other  hand,  is 
a  one-man  skill.  Realistic  flight  games  such  as  FlightSim  could  provide 
individual  proficiency). 

•  The  focus  of  the  gaming  effort  should  be  on  determining  what  “tasks”  could 
best  be  trained  through  PC  games.  Cues  and  responses  for  specific  tasks  can 
be  determined,  then  a  determination  made  about  the  game’s  capability  to 
provide  the  required  inputs.  They  mentioned  the  TPSC  codes  used  with  the 
CCTT  studies. 
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•  A  method  is  needed  to  list,  by  task,  what  the  developer  has  to  provide  to 
support  training. 

•  An  alternative  is  to  use  Tac  Ops,  a  PC  game  recently  introduced  at  USAARMS 
for  armor  branch  advanced  course  students.  The  game  is  apparently 
somewhat  “primitive”  but  has  utility  for  armor  company  grade  officers  (LTs  and 
CPTs). 
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Sample  Game  Maps  -  Steel  Beasts1"1 
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Steel 


Kpy: 


Options /Details 


Extra  hformatjon 


Predefined  Information 


Blue  Scoring 
Condition 


Red  Scorhg 
Con  cH  ion 


**  Select  Multiplayer  Mission 
Austin  Recon 

Recon  town  of  Austin 
Convoy  Escort  (MP) 

Escort  hummers  across  an  island 
Deliberate  Assault  01  MP 

■Steze  objectives  VHrlrg  a  Mongol 
Dei  berate  Asset*  01  W*  CO-OP 

Steze  objectives  Viking  a  Mongol 
Defcerate  Asset*  01  MP  Small 

Slexe  objectives  Viking  aMongoi 
Delierats  Assault  02  MP 

Steze  town  at  objective  Jupter 
Steze  crossroads  and  bridges  at  objective  Mars 
Deliberate  Assaut  02  MP  CO-OP 

Steze  town  at  objective  Jupter 
Steze  crossroads  and  bridges  at  objective  Mars 
Leo2A4  VSM1A1  MP 

Defend  wth  Leopards  against  Ml  ass  out 
Leo  2A4  vsMIAl  MP  2 

Defend  wth  Leopards  against  Ml  assault 
Leo2A4vsMlA1  MP3 

Defend  wth  Leopards  against  Ml  assault 
Ml  v  Loo2  2on2  Death  Match 
Ml  v  Lec2  2on2  Sm!  Death  Match 
Ml  v  Leo2  2on2  Wth  FVs 
Ml  v  Leo2  4oo4  Death  Match 
Ml.  V  Loo2  40TS4  Wth  IFW 
Ml  v  Leo2  6on6  Death  Match 
Ml  vLeo2  6on6VMth(FVs 
Ml  vLeo2  8on8  Death  Match 
Ml  vLeo2  6on0WthFVs 
Ml  vLeo2  Death  Match 
Meeting  In  The  River  Valy  MP 
Meeting  Vi  The  River  Voly  pt.2  MP 
MPLeo2s  Asset*  Mis 

Defend  with  Ml  a  against  Leopard  assault 
MP  Ml  *  Asset*  Le02s 

Defend  wth  Leopards  against  Ml  asset* 

MP  OPFOR  Assaults  Leo2s 

Defend  wth  Leopards  against  Soviet  assault 
MP  OPFOR  Assaults  Mis 

Defend  wth  Mis  against  Soviet  as saut 
MTC  In  The  Desert  01  W* 

Steze  objectives  Ramste  &  Isis 
MTC  In  The  Desert  01  MP  Small 

Steze  objectives  Rams  is  &  isis 
Tar*  Table  XII  iv*> 

Platoon  leader  qualification  course 


■“  Elements: 
Dirt 
Grass 
Sand 
Water 
Bog 
Woods 
Tr»» 

Bush 

Rood 

Ofct 

Baml  (object) 
Barn2  (object) 


"Support 
avaiabte  n 
none,  low, 
medum,hi£i, 
and  top 


E-4 


E-5 


Send  Action  Mission- 
A  Vltese  Celled  Netreba 

Defend  vfltage  from  attack 

Ambush 

Defeat  the  enemt  recon  unis 
Are  They  Attacking  Here 

Correct [y  determine  the  nahre  and  intent  of  the  enemy  force 
Defeat  enemy  forces 

BJjbovsMI 

Defend  against  Ml  ossaut  w*h  Leopard 

B_Kecon 

Recon noiter  the  vicinity  of  a  bridge 
Battle  in  Kuwait 

Begin  ground  attack  against  kaqi  forces 
Battle  Of  The  Wiese  Valley 

Defeat  the  enemy’s  recon  forces  &  hat  southern  advoncem 
Breectenbastef 

Explore  territory 
Bridgehead 

Retake  a  captured  bridge 
Canadton_Frost 

Defend  territory 
Calvary  To  The  Rescue 

Guard  A/1  -6  Infantry 
Combat  Trains 

Destroy  aft  enemy  forces 
Company  Convoy 

Friendly  forces  must  reach  TAA  VAN  HALEN  and  secure  I 
Detfeerate  Assault  01 

Steze  objectives  Viking  and  Mongol 
etiberate  Assaut  02 

Steie  town  at  objective  Jupiter 
Sfeze  bridges  end  crossroads  at  objective  Mars 
rate  Defense  01 
Defend  BP1  &  BP2 
.Attack 

Take  objective  before  reinforcements  can  arrive 
mo.AttackJ.eo 

Attack  the  enemy  company  and  ct t  off  reinforcements  A 

wt  Attack 

Break  throutfithe  Motor  Rifle  Division  In© 
sert  Detey3 

Deiey  advancing  troops 
.....  From  Dozen 

Escape  from  behind  enemy  fines 
FH01  First  Contact 

take  out  the  T-8Gs 
FH02  First  Attack 

Capture  06J-1  SOBJf-3 
FH03  Flank  Guard 

Defend  bridge 


Fort  Apache 

Rescue  surviving  unks  of  a  US*ted  armored  patrol 
GenJ?econ1 

Reconhoter  enemy  positions  at  two  bridges 
Hasty  Defense  01 

Defend  against  advancing  forces 
Heavy  Reconnaissance 

Recomoier  area  in  preparaion  for  battle 
instart  Action  -  Ambush 

Two  Ml  s  ambush  a  company  of  Opfor  tanks  and  BMPs 
Instant  Action  -  Rood  March 

Cope  with  being  ambushed 
Instant  Action  -  Urban  Assau* 

Blue  is  launching  an  attack  on  a  smal  town  and  Red  is  defendkig 
JRS  -  Korean  Mghtmera  2 

Hold  the  me  at  Seoul 
Korean  Nightmare 

Defend  against  advancing  forces 

Assist  a  company  In  spearheading  your  betaion’s  march 

Meeting  2 

intercept  enemy  tanks 
Movement  To  Contact  01 

Attack  enemy  unfts  defending  abjective  Atlas 

MO  Vkig_ Ambush 

Practice  executing  a  moving  ambush 
Operation  Dust  Devil 

End  enemy  agression  at  the  Iraq /Kuwait  border 
Operation  Sieve 

Act  My  stop  raids  before  forces  reach  Namibian  territory 
Storm  in’  The  Desert 

Qprrafoi  DwfitSbm 
Sucker  Punch  In  Kuwa* 

Attack  Iraqi  forces  under  veil  of  dust  storm 

Tank  Table  Vli 

Test  engagement  of  moving  and  stationary  targets 
Tank  Table  VI 3 

Test  engagement  dt  moving  and  stationery  targets 
Tank  Table  VI  •  4 

Test  engagement  of  moving  and  stationary  targets 
Tank  Table  VI 

Ratoon  leader  cjuaiffcation  course 

anks 

Assault  on  tanks 
The  Battle  For  Windy  River 

Secure  a  crucial  river  crosshg 
The  Siege  Of  Dozen 

Take  Dozen 
The  Warts  Objective 

'  Rec'conoler  enemy  forces 

Plan  arid  conduct  an  attack  on  enemy  forces 
Wfeods_Have_Eyee 

Slow  enemy  advancements  unti  reinforcements  arrive 
A  Translated  from  German... 
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